Anomaly Detector For Vehicle Control Signals

ABSTRACT

A vehicle controller receives sensor outputs and identifies features in the sensor outputs. The controller determines a trajectory based on the features and generates control signals to vehicle actuators to follow the trajectory. Eccentricity of the control signals is evaluated and if it meets a threshold condition is met an intervention is performed such as discarding or modifying the control signal or initiating a safety procedure. Eccentricity may be determined using an unsupervised machine learning model. The threshold condition may be a dynamic threshold condition such as using the n-sigma approach or the Chebyshev inequality.

CROSS REFERENCE TO RELATED PATENT APPLICATION(S)

The present disclosure is part of a continuation application of U.S.patent application Ser. No. 16/031,702, filed on Jul. 10, 2018, thecontent of which is incorporated by reference in its entirety.

BACKGROUND Field of the Invention

This invention relates to control systems of autonomous vehicles.

Background of the Invention

Autonomous vehicles are an area of intense research and development. Thecontroller of an autonomous vehicle typically receives outputs ofvarious sensors such as cameras, RADAR sensors, LIDAR sensors, or thelike. These outputs are then processed to identify road surfaces, laneboundaries, other vehicles, pedestrians, and potential obstacles. Thistypically requires processing a large amount of data in a very shortamount of time in order to properly react to changing conditions.

It would be an advancement in the art to improve the safety ofautonomous vehicles.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of the invention will be readilyunderstood, a more particular description of the invention brieflydescribed above will be rendered by reference to specific embodimentsillustrated in the appended drawings. Understanding that these drawingsdepict only typical embodiments of the invention and are not thereforeto be considered limiting of its scope, the invention will be describedand explained with additional specificity and detail through use of theaccompanying drawings, in which:

FIG. 1 is a schematic block diagram of components implementing a vehiclein accordance with an embodiment of the present invention;

FIG. 2 is a schematic block diagram of an example computing devicesuitable for implementing methods in accordance with embodiments of theinvention;

FIG. 3 illustrates components for generating control signals accordingto sensor outputs in accordance with an embodiment of the presentinvention; and

FIG. 4 is a process flow diagram of a method for detecting anomalies incontrol signals in accordance with an embodiment of the presentinvention.

DETAILED DESCRIPTION

Referring to FIG. 1, a vehicle may include any vehicle known in the art.The vehicle may have all of the structures and features of any vehicleknown in the art including, wheels, a drive train coupled to the wheels,an engine coupled to the drive train, a steering system, a brakingsystem, and other systems known in the art to be included in a vehicle.

As discussed in greater detail herein, a controller 102 mounted in thevehicle may perform autonomous navigation and collision avoidance. Thecontroller 102 may receive one or more outputs from one or more exteriorsensors 104. For example, one or more cameras 106 a may be mounted tothe vehicle and output image streams received to the controller 102. Insome embodiments, the exterior sensors 104 combined provide a 360 degreeview around the vehicle. In other embodiments, a smaller viewing anglemay be achieved by the exterior sensors 104, such as an angle between180 and 270 degrees.

The exterior sensors 104 may include sensors such as an ultrasonicsensor 106 b, a RADAR (Radio Detection and Ranging) sensor 106 c, aLIDAR (Light Detection and Ranging) sensor 106 d, a SONAR (SoundNavigation and Ranging) sensor 106 e, and the like.

The controller 102 may execute an autonomous operation module 108 thatreceives the outputs of the exterior sensors 104. The autonomousoperation module 108 may include an obstacle identification module 110a, a collision prediction module 110 b, and a decision module 110 c. Theobstacle identification module 110 a analyzes the outputs of theexterior sensors and identifies potential obstacles, including people,animals, vehicles, buildings, curbs, and other objects and structures.In particular, the obstacle identification module 110 a may identifyvehicle images in the sensor outputs.

The collision prediction module 110 b predicts which obstacle images arelikely to collide with the vehicle based on its current trajectory orcurrent intended path. The collision prediction module 110 b mayevaluate the likelihood of collision with objects identified by theobstacle identification module 110 a. The decision module 110 c may makea decision to stop, accelerate, turn, etc. in order to avoid obstacles.The manner in which the collision prediction module 110 b predictspotential collisions and the manner in which the decision module 110 ctakes action to avoid potential collisions may be according to anymethod or system known in the art of autonomous vehicles.

The decision module 110 c may control the trajectory of the vehicle byactuating one or more actuators 112 controlling the direction and speedof the vehicle. For example, the actuators 112 may include a steeringactuator 114 a, an accelerator actuator 114 b, and a brake actuator 114c. The configuration of the actuators 114 a-114 c may be according toany implementation of such actuators known in the art of autonomousvehicles.

The decision module 110 c or another module may control the actuators112 according one or more other algorithms, such as an anti-lock brakingsystem, stability control system, traction control system, or othervehicle system.

In embodiments disclosed herein, the autonomous operation module 108 mayperform autonomous navigation to a specified location, autonomousparking, and other automated driving activities known in the art.

In some embodiments, outputs of the decision module to the actuators 112may be evaluated for anomalies as described hereinbelow. Other outputscontrolling other vehicle systems (fuel delivery, engine, suspension,etc.) may also be evaluated for anomalies in the same manner.Accordingly, the autonomous operation module 108 may further include ananomaly detection module 110 d that performs anomaly detection asdescribed below.

FIG. 2 is a block diagram illustrating an example computing device 200.Computing device 200 may be used to perform various procedures, such asthose discussed herein. The controller 102 may have some or all of theattributes of the computing device 200.

Computing device 200 includes one or more processor(s) 202, one or morememory device(s) 204, one or more interface(s) 206, one or more massstorage device(s) 208, one or more Input/Output (I/O) device(s) 210, anda display device 230 all of which are coupled to a bus 212. Processor(s)202 include one or more processors or controllers that executeinstructions stored in memory device(s) 204 and/or mass storagedevice(s) 208. Processor(s) 202 may also include various types ofcomputer-readable media, such as cache memory.

Memory device(s) 204 include various computer-readable media, such asvolatile memory (e.g., random access memory (RAM) 214) and/ornonvolatile memory (e.g., read-only memory (ROM) 216). Memory device(s)204 may also include rewritable ROM, such as Flash memory.

Mass storage device(s) 208 include various computer readable media, suchas magnetic tapes, magnetic disks, optical disks, solid-state memory(e.g., Flash memory), and so forth. As shown in FIG. 2, a particularmass storage device is a hard disk drive 224. Various drives may also beincluded in mass storage device(s) 208 to enable reading from and/orwriting to the various computer readable media. Mass storage device(s)208 include removable media 226 and/or non-removable media.

I/O device(s) 210 include various devices that allow data and/or otherinformation to be input to or retrieved from computing device 200.Example I/O device(s) 210 include cursor control devices, keyboards,keypads, microphones, monitors or other display devices, speakers,printers, network interface cards, modems, lenses, CCDs or other imagecapture devices, and the like.

Display device 230 includes any type of device capable of displayinginformation to one or more users of computing device 200. Examples ofdisplay device 230 include a monitor, display terminal, video projectiondevice, and the like.

Interface(s) 206 include various interfaces that allow computing device200 to interact with other systems, devices, or computing environments.Example interface(s) 206 include any number of different networkinterfaces 220, such as interfaces to local area networks (LANs), widearea networks (WANs), wireless networks, and the Internet. Otherinterface(s) include user interface 218 and peripheral device interface222. The interface(s) 206 may also include one or more peripheralinterfaces such as interfaces for printers, pointing devices (mice,track pad, etc.), keyboards, and the like.

Bus 212 allows processor(s) 202, memory device(s) 204, interface(s) 206,mass storage device(s) 208, I/O device(s) 210, and display device 230 tocommunicate with one another, as well as other devices or componentscoupled to bus 212. Bus 212 represents one or more of several types ofbus structures, such as a system bus, PCI bus, IEEE 1394 bus, USB bus,and so forth.

For purposes of illustration, programs and other executable programcomponents are shown herein as discrete blocks, although it isunderstood that such programs and components may reside at various timesin different storage components of computing device 200, and areexecuted by processor(s) 202. Alternatively, the systems and proceduresdescribed herein can be implemented in hardware, or a combination ofhardware, software, and/or firmware. For example, one or moreapplication specific integrated circuits (ASICs) can be programmed tocarry out one or more of the systems and procedures described herein.

FIG. 3 illustrates an example flow of control in an autonomous vehiclethat includes anomaly detection according to the methods disclosedherein. For example, the exterior sensors 104 or other sensor sensing astate of the vehicle (engine sensors, drive train sensors, yaw ratesensor, roll sensor, etc.) provide their outputs to perception modulesthat detect features in the sensor outputs, such as obstacles, roadsurfaces, other vehicles, pedestrians, traffic lights, signs, etc. Forexample, the perception modules may perform the functions ascribed tothe obstacle identification module 110 a and collision prediction module110 b.

The features as detected by the perception modules 110 a, 110 b areinput to a control algorithm that determines a trajectory of the vehicleaccording to the detected features. For example, the control algorithmmay perform some or all of the functions ascribed to the decision module110 c.

The output of the decision module 110 c are control signals to thevehicle actuators 112 or other systems of the vehicle. These controlsignals are monitored by the anomaly detection module 110 d, whichattempts to identify anomalies in the control signals that have thepotential to cause passenger discomfort or otherwise compromise thesafety of passengers of the vehicle.

Since the vast majority of the perception tasks (e.g. pedestrian andvehicle detection/tracking, lane marking detection, traffic lightdetection) are performed with the aid of some type of neural network,which may have difficulty handling uncertainty, the anomaly detectionmodule 110 d evaluates control signals (e.g. acceleration, braking,steering) generated by the control algorithm to detect anomalies, suchas those that might be caused by anomalies in the perception tasks. Inthe illustrated approach, this is performed by evaluating the controlsignals to the actuators 112. In other embodiments, anomaly detection isperformed by analyzing a data stream at another stage in the process ofperception and generating control signals.

In this manner, the anomaly detection module 110 d handles automateddriving scenarios that might otherwise result in abrupt changes in thecontrol signals, such as abrupt braking, abrupt acceleration or abruptsteering, unless such abrupt changes strictly required.

Referring to FIG. 4, the illustrated method 400 may be executed by thecontroller 102 of a vehicle, such as the autonomous vehicle of FIG. 1using the anomaly detection module 110 d.

The method 400 include receiving 402 sensor outputs from the exteriorsensors 104 or any of the other vehicle sensors discussed above.

The method 400 further include detecting 404 features in the sensoroutputs such as obstacles, road surfaces, road signs, pedestrians, othervehicles, or the like. Vehicle control signals are then generated 406for actuators 112 of the vehicle in order to traverse a trajectorydetermined according to the features. The control signals may begenerated 406 according to any approach for controlling an autonomousvehicle known in the art. As noted above, the control signals mayfurther include control signals for other vehicle systems as notedabove.

The method 400 may further include calculating 408 eccentricity of thecontrol signals. In particular, a set of control signals overtime may beinput to an unsupervised machine learning algorithm that is therebytrained over time to determine when values in the set of control signalsare anomalous. Note that this may include either of (a) determining thatone or more values of a particular control signal is anomalous or (b)that the co-occurrence of a set of values for multiple control signalscollectively is anomalous either with or without the value of anyparticular control signal being anomalous.

The unsupervised machine learning model may be trained to identifyanomalies according to any approach known in the art. For example, thismay include the approach for identifying anomalies in an arbitrarynumber of signal as described in P. Angelov, “Anomaly detection based oneccentricity analysis,” 2014 IEEE Symposium on Evolving and AutonomousLearning Systems (EALS), Orlando, Fla., 2014, pp. 1-8, which is herebyincorporated herein by reference.

Any signals that are read in a data stream form (one sample per timeinstant) may be used in the eccentricity calculation. Hence, any controlsignal applied to the vehicle can be monitored by the method disclosedin this document. In some embodiments, the eccentricity calculation isonly performed for those controls signals that, when changed abruptly,can cause discomfort to the passengers or even put their safety at risk.Therefore, signals such as acceleration, braking and steering controlsare good candidates for this purpose.

The method 400 may include evaluating 410 one or both of the controlsignals and the eccentricity as determined at step 408 with respect to athreshold. The threshold may be a static threshold or a dynamicthreshold. For example, the threshold may be a dynamic threshold basedon past values for the eccentricity during a training period. Forexample, the “n-sigma rule” approach may be used, where n is specifiedby an operator. In another example, the Chebyshev inequality approach isused to determine a dynamic threshold, such as is described in J. G.Saw, M. Yang, and T. C. Mo, “Chebyshev inequality with estimated meanand variance,” The American Statistician, vol. 38, no. 2, pp. 130-132,1984, which is hereby incorporated by reference in its entirety.

If the threshold condition is found to be met 412, the method 400 mayinclude intervening. In a simple case, an alert is generated. In a morecomplex case, the control signals are modified in response to thethreshold condition being met. For example, a control signal that isanomalous may be modified to be within a predefined boundary. In anotherexample, the intervention includes substituting 414 pre-defined fallbacksignals for the control signals and inputting the fallback signals tothe actuators 112 or other vehicle systems. In another example, in theevent that an anomaly is detected, the monitor can either discard acontrol signal, if that is an outlier, or initiate a safety procedure,if the abrupt deviation persists, such as decelerating to a safe speedor bringing the vehicle to a safe stop.

The control signals, either original or substituted according to step414, may be transmitted 416 to their corresponding actuators 112, whichthen alter their states according to the control signals.

Note that the method 400 is performed “in real time.” That is to saythat the eccentricity analysis may be performed as the control signalsare generated but before they are input to the corresponding actuators112 or other vehicle systems. For example, the eccentricity analysis andany intervention may be performed within 0.1 to 20 milliseconds of thetime control signals are generated and available at 408.

-   -   In the above disclosure, reference has been made to the        accompanying drawings, which form a part hereof, and in which is        shown by way of illustration specific implementations in which        the disclosure may be practiced. It is understood that other        implementations may be utilized and structural changes may be        made without departing from the scope of the present disclosure.        References in the specification to “one embodiment,” “an        embodiment,” “an example embodiment,” etc., indicate that the        embodiment described may include a particular feature,        structure, or characteristic, but every embodiment may not        necessarily include the particular feature, structure, or        characteristic. Moreover, such phrases are not necessarily        referring to the same embodiment. Further, when a particular        feature, structure, or characteristic is described in connection        with an embodiment, it is submitted that it is within the        knowledge of one skilled in the art to affect such feature,        structure, or characteristic in connection with other        embodiments whether or not explicitly described.

Implementations of the systems, devices, and methods disclosed hereinmay comprise or utilize a special purpose or general-purpose computerincluding computer hardware, such as, for example, one or moreprocessors and system memory, as discussed herein. Implementationswithin the scope of the present disclosure may also include physical andother computer-readable media for carrying or storingcomputer-executable instructions and/or data structures. Suchcomputer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arecomputer storage media (devices). Computer-readable media that carrycomputer-executable instructions are transmission media. Thus, by way ofexample, and not limitation, implementations of the disclosure cancomprise at least two distinctly different kinds of computer-readablemedia: computer storage media (devices) and transmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM,solid state drives (“SSDs”) (e.g., based on RAM), Flash memory,phase-change memory (“PCM”), other types of memory, other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store desired program code means inthe form of computer-executable instructions or data structures andwhich can be accessed by a general purpose or special purpose computer.

An implementation of the devices, systems, and methods disclosed hereinmay communicate over a computer network. A “network” is defined as oneor more data links that enable the transport of electronic data betweencomputer systems and/or modules and/or other electronic devices. Wheninformation is transferred or provided over a network or anothercommunications connection (either hardwired, wireless, or a combinationof hardwired or wireless) to a computer, the computer properly views theconnection as a transmission medium. Transmissions media can include anetwork and/or data links, which can be used to carry desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer. Combinations of the above should also be includedwithin the scope of computer-readable media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. The computerexecutable instructions may be, for example, binaries, intermediateformat instructions such as assembly language, or even source code.Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, an in-dash vehicle computer, personalcomputers, desktop computers, laptop computers, message processors,hand-held devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, mobile telephones, PDAs, tablets, pagers, routers, switches,various storage devices, and the like. The disclosure may also bepracticed in distributed system environments where local and remotecomputer systems, which are linked (either by hardwired data links,wireless data links, or by a combination of hardwired and wireless datalinks) through a network, both perform tasks. In a distributed systemenvironment, program modules may be located in both local and remotememory storage devices.

Further, where appropriate, functions described herein can be performedin one or more of: hardware, software, firmware, digital components, oranalog components. For example, one or more application specificintegrated circuits (ASICs) can be programmed to carry out one or moreof the systems and procedures described herein. Certain terms are usedthroughout the description and claims to refer to particular systemcomponents. As one skilled in the art will appreciate, components may bereferred to by different names. This document does not intend todistinguish between components that differ in name, but not function.

It should be noted that the sensor embodiments discussed above maycomprise computer hardware, software, firmware, or any combinationthereof to perform at least a portion of their functions. For example, asensor may include computer code configured to be executed in one ormore processors, and may include hardware logic/electrical circuitrycontrolled by the computer code. These example devices are providedherein purposes of illustration, and are not intended to be limiting.Embodiments of the present disclosure may be implemented in furthertypes of devices, as would be known to persons skilled in the relevantart(s).

At least some embodiments of the disclosure have been directed tocomputer program products comprising such logic (e.g., in the form ofsoftware) stored on any computer useable medium. Such software, whenexecuted in one or more data processing devices, causes a device tooperate as described herein.

While various embodiments of the present disclosure have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. It will be apparent to persons skilledin the relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the disclosure.Thus, the breadth and scope of the present disclosure should not belimited by any of the above-described exemplary embodiments, but shouldbe defined only in accordance with the following claims and theirequivalents. The foregoing description has been presented for thepurposes of illustration and description. It is not intended to beexhaustive or to limit the disclosure to the precise form disclosed.Many modifications and variations are possible in light of the aboveteaching. Further, it should be noted that any or all of theaforementioned alternate implementations may be used in any combinationdesired to form additional hybrid implementations of the disclosure.

1. A method comprising, by a vehicle controller: receiving sensoroutputs from sensors of the vehicle; processing the sensor outputs by acontrol algorithm to obtain control outputs for the vehicle; evaluatingeccentricity of the control outputs collectively with respect to athreshold condition; and when the control outputs meet the thresholdcondition, generating an intervention.
 2. The method of claim 1, whereinprocessing the outputs by the control algorithm comprises: detectingfeatures in the sensor outputs; selecting a trajectory according to thefeatures; and generating the control outputs effective to traverse thetrajectory.
 3. The method of claim 1, wherein the control outputscomprise control signals to a steering actuator, a braking actuator, andan accelerator actuator.
 4. The method of claim 1, wherein the controlalgorithm comprises at least one of an anti-lock braking system, astability control system, and a traction control system.
 5. The methodof claim 1, wherein the sensor outputs comprise outputs of at least oneof a radio detection and ranging (RADAR) sensor, a light detection andranging (LIDAR) sensor, and a camera.
 6. The method of claim 1, whereinevaluating the eccentricity of the control outputs with respect to thethreshold condition comprises evaluating the eccentricity metric overtime with respect to a Chebyshev inequality.
 7. The method of claim 1,wherein the intervention comprises generating an alert.
 8. The method ofclaim 1, wherein the intervention comprises substituting fallbackcontrol signals for the control outputs.
 9. A vehicle comprising: aplurality of sensors; a plurality of actuators for controlling operationof the vehicle; a controller coupled to the plurality of sensors and theplurality of actuators, the controller being programmed to: receivingsensor outputs from the plurality of sensors; process the sensor outputsby a control algorithm to obtain control outputs for the plurality ofactuators; evaluating eccentricity of the control outputs with respectto a threshold condition; and when the control outputs meet thethreshold condition, generate an intervention.
 10. The vehicle of claim9, wherein the controller is further programmed to process the outputsby the control algorithm by: detecting features in the sensor outputs;selecting a trajectory according to the features; and generating thecontrol outputs effective to traverse the trajectory.
 11. The vehicle ofclaim 9, wherein the plurality of actuators comprise a steeringactuator, a braking actuator, and an accelerator actuator.
 12. Thevehicle of claim 9, wherein the control algorithm comprises at least oneof an anti-lock braking system, a stability control system, and atraction control system.
 13. The vehicle of claim 9, wherein theplurality of sensors comprise at least one of a radio detection andranging (RADAR) sensor, a light detection and ranging (LIDAR) sensor,and a camera.
 14. The vehicle of claim 9, wherein the controller isprogrammed to evaluate the eccentricity of the control outputs withrespect to the threshold condition by inputting the control outputs intoan unsupervised machine learning model to obtain an eccentricity metric.15. The vehicle of claim 9, wherein the intervention comprisesgenerating an alert.
 16. The vehicle of claim 9, wherein theintervention comprises substituting fallback control signals for thecontrol outputs.